In the advanced assisted driving device, machine vision technology was used to process the video of vehicles in front in real time to dynamically recognize and predict the posture and behavior of vehicle. Concerning low precision and large delay of this kind of recognition algorithm, a deep learning algorithm for vehicle behavior dynamic recognition based on Long Short-Term Memory (LSTM) was proposed. Firstly, the key frames in vehicle behavior video were extracted. Secondly, a dual convolutional network was introduced to analyze the feature information of key frames in parallel, and then LSTM network was used to sequence the extracted characteristic information. Finally, the output predicted score was used to determine the behavior type of vehicle. The experimental results show that the proposed algorithm has an accuracy of 95.6%, and the recognition time of a single video is only 1.72 s. The improved dual convolutional network algorithm improves the accuracy by 8.02% compared with ordinary convolutional network and increases by 6.36% compared with traditional vehicle behavior recognition algorithm based on a self-built dataset.